In one example, a complex task includes a set of data-dependent, stand-alone applications, along with their various connections (e.g., pipes and links), which are used to process various pieces of data to complete a specific task or job. The components of a complex task (e.g., the applications, pipes and links) may be distributed across multiple computer systems of a computer environment or reside within one computer system.
In typical distributed computer environments of today, a complex task may be run across a network of heterogeneous operating systems. Specifically, a user individually invokes remote executions of various scripts/commands to run the complex task across the network of heterogeneous operating systems.
The user is faced with inherent management complexities when invoking the numerous applications over the several hosts and with communicating with different operating systems. This task has proved to be cumbersome, time-consuming and error prone.
For example, configuration errors can manifest themselves when the user provides insufficient or incorrect data to invoke the various components of the complex task. Data flow errors can occur if the output of one program does not properly transfer to become the input of another program. Data connection mismatches can occur if the characteristics of the data (e.g., name, host, etc.) are changed, but not propagated throughout the affected components of the complex task. Further, run sequence problems can occur when certain components must be run before others.
The detection of the errors has also proved to be problematic, since the errors do not surface until run-time.
Thus, a need exists for a centralized control for managing a complex task. A further need exists for a complex task management capability that offers pre-run dynamic validation of the various components of a complex task, programmatic determining of the proper order to start, stop and/or delete a component, post-run automatic recovery from a terminated component, and automatic collection of information regarding the various components, such that the overall status of a complex task is obtained and understood.